Dual pixel pitch imaging array with extended dynamic range

ABSTRACT

An integrated focal plane provides two co-aligned, overlapping pixel arrays in two formats, one with large pixels and low pixel count, the other with small pixels and high pixel count. Typically, the large pixels are 10 to 100 times larger in area than the small pixels. The dual arrays are disposed in a single detector substrate flip-chip bonded to a single readout circuit. They are sensitive to two infrared colors, one shorter and one longer wavelength band. The dual array focal plane concurrently provides two distinct pixel instantaneous fields of view within the same overall field of view as well as simultaneous fast and slow frame rates. The dual frame rates allow for combined fast sensing with sensitive imaging. Differing spatial and temporal data enables enhanced image processing for improved clutter rejection and detection performance. Differing gains combined with the dual frame rates provide an extended dynamic range.

This application claims the priority of U.S. patent application Ser. No.13/625,493 filed Sep. 24, 2012, which claims the priority of U.S.provisional patent application No. 61/538,064 filed Sep. 22, 2011.

FIELD OF THE INVENTION

The present invention generally relates to camera focal plane imagingarrays. More specifically, the invention relates to a camera focal planeimaging array that is optimized to meet competing performancerequirements for military or security sensing systems.

BACKGROUND OF THE INVENTION

For military-use or security-use systems, multiple competingrequirements complicate sensor design, especially infrared (IR) sensordesign. Quantitatively, an ideal military/security system needs a fastsensing mode that operates at maximum rates of 1500 to 5000 frames persecond, combined with a slow, staring capability that can match thesensitivity of a high-quality 30 frame per second sensor. It needsresolution suitable for fine-grained images, and a dynamic range (thedifference between the darkest signal that can be measured and thebrightest signal that can be measured) of over one million times (10million or more is optimal). It should provide new types of raw datathat enable qualitative improvements in anti-clutter techniques, thatis, techniques to distinguish between threatening events and othermanmade events of no interest (non-threatening events).

No detector technology exists that meets these goals. Systems aretypically customized to support a specific requirement; for example,they are able to function in an optically bright tactical militaryenvironment or an optically dark environment, but not both. Or, asanother example, they are optimized for speed by sacrificing resolution,or optimized for resolution by sacrificing speed.

Design approaches employed today focus primarily on enhanced resolution,i.e. more pixels and smaller pixels. This is important, but it onlyaddresses one aspect of the multiple critical needs outlined above. Asecond area of design emphasizes “all digital” sensor chips whereinsignal digitization is performed on the sensor readout circuit insteadof off the sensor chip assembly. Some of these designs have thepotential to address the need for larger dynamic range.

When higher data processing speeds are required, either a designtradeoff is made —fewer pixels for less resolution and therefore lessdata to be processed—or output channels are added to enable the higherspeed. For example, rather than using two or four pixel data signallines, eight or sixteen could be used. This achieves a 2× to 8× speedimprovement, but adds complexity to the external chip controlelectronics and to the camera mechanical housing, which is typically acryogenic dewar flask. An improvement of 25× to 50× is what is reallyrequired (for example, from 60 frames per second to 2000 frames persecond or greater).

Current IR sensors achieve a dynamic range of less than 20 thousand, andthis often presents a problem in real applications. In comparison, thehuman eye can see a total brightness range of over one million times. Invisible photography, systems have been produced that achieve a dynamicrange of over one million—these are called HDR for High Dynamic Range.Also, so-called sigma-delta digital designs, if implemented with 20 ormore encoding bits, have the potential for a dynamic range of onemillion. However, both HDR and the sigma-delta approach result inperformance tradeoffs. A different approach is used here.

Clutter rejection methods have employed two-color radiometrictechniques, which are highly effective, but clutter in the IR bandsremains a difficult problem. Two-color methods typically compare thebrightness in two IR bands to determine the approximate temperaturerange of an object or event. The temperature information is then used todetermine whether an observed event is threatening, such as the launchof a shoulder-fired missile, or benign, such as the glint of the sun offa car windshield. However, these techniques are limited by how preciselythe temperature is determined. They are also limited because some benignevents have similar temperatures to threatening ones. For example, itwould be difficult to distinguish a gunshot from an automobile backfireusing temperature alone. Achieving better results in this area requiresthe equivalent of combining multiple sensor data with the two-colorradiometry data.

Achieving the very large dynamic range of operation required formilitary/security systems, combined with flexible operation, whileproviding new types of data for clutter rejection is a very difficultchallenge. Simply extending current design approaches is not the answer.A new approach is needed.

In part, the inspiration for a sensor that can address the above needscan be drawn from biology and the human eye. The eye combines two typesof sensors, rods and cones, each of which is optimized for differentsensing regimes. The eye and neural pathways also allow for multiplemodes of sensing, including fast reactions and “normal” scanning of ascene for more detail, which is significantly slower. Finally, the eyeextends its dynamic range with the pupil, which provides fast dim/brightswitching based on overall scene brightness.

Accordingly, it is an object of the present invention to provide asensor that can sense very high-speed events, typically associated withweaponry. Examples include sensing projectiles such as bullets; trackingsupersonic maneuvering missiles; sensing the launch flash from theejection of a projectile from a weapon; and detecting gunshot muzzleflashes.

It is another object of the present invention to provide a sensor withthe ability to support a high (large) dynamic range, i.e. the ability tosimultaneously sense dim or low contrast objects along with bright orhigh contrast objects. Examples of dim/low contrast objects include coldobjects in space, or concealed personnel against a warm earthbackground. Examples of bright or high contrast objects include gunfireor explosions.

It is a further object of the present invention to provide a sensor thatcan avoid saturation for very bright events, such as large explosions.If a sensor is saturated, it is temporarily blinded.

It is still a further object of the present invention to provide asensor with the ability to produce a high-resolution image that showsdetail.

It is still a further object of the present invention to provide datathat can be used to improve methods to avoid false alarms from manmadesignals or clutter, which typically limit sensor performance more thansignal to noise for non-space applications.

The following patents may be relevant to the field of the invention:

U.S. Pat. No. 7,608,823 to Tennant, incorporated herein by reference,discloses a multimode focal plane array architecture with electricallyisolated commons for independent sub-array biasing to accommodate largebias amplitude differences and different temporal bias profiles.

U.S. Pat. No. 7,075,079 to Wood, incorporated herein by reference,discloses a dual wavelength focal plane having a first array of infraredsensing pixel elements, and a second array of visible light pixelelements adapted to be selective to colors encountered while driving anautomobile. The arrays are vertically stacked on a monolithic siliconsubstrate, and they are electrically coupled to a processor and displayto integrate the infrared and color pixel elements into a view for adriver of the automobile.

U.S. Pat. No. 6,407,439 to Hier et al., incorporated by reference,discloses a plurality of photodetectors, each sensitive to differentwavelengths, integrated on a common semiconductor substrate. Thedifferent photodetectors can be stacked over one another or placedlaterally on the common substrate. Three of such photodetectors can forma pixel of an active matrix array for an image sensor. The differentphotodetectors in each pixel can be multiplexed electronically. Theelectronic circuits for activating the different photodetectors can beintegrated on the same substrate.

U.S. Pat. No. 6,034,407 to Tennant, incorporated herein by reference,discloses multi-spectral planar photodiode pixels for simultaneouslydetecting multi-colors of infrared radiation. Each multi-spectral planarphotodiode pixel includes a semiconductor substrate layer, a bufferlayer of a first conductivity type material deposited on a semiconductorsubstrate layer, and a first color layer of the first conductivity typematerial deposited on the buffer layer. The multi-spectral planarphotodiode pixel further includes a barrier layer of the firstconductivity type material deposited on the first color layer, a secondcolor layer of the first conductivity type material deposited on thebarrier layer, and a cap layer of the first conductivity type materialdeposited on a the second color layer. A first diode comprising of asecond conductivity type material is formed in the first color layer,and a second diode comprising a second conductivity type material isformed in the second layer.

U.S. Pat. No. 5,903,659 to Kilgore, incorporated herein by reference,discloses an adaptive method for removing fixed pattern noise from focalplane array (FPA) imagery (sensor images). A set of correction terms isapplied to a blurred version of the FPA image, and a filter is appliedto the corrected, blurred image. Fixed pattern noise errors are thencalculated using the filtered imagery, and employed to update thecorrection terms. The updated correction terms are then used forprocessing the next image. In one embodiment, the filter is ananti-median filter. In another embodiment, the filter is an anti-meanfilter. These methods are commonly referred to as non-uniformitycorrection, or NUC.

U.S. Pat. No. 5,751,049 to Goodwin, incorporated herein by reference,discloses a two-colored infrared detector comprising elements having oneor more diodes and a metal insulator semiconductor (MIS) device. Thediodes are comprised of regions of semiconductor materials, which areoperable to generate electron-hole pairs when struck by infraredradiation having first and second wavelengths. The capacitor includes agate which is operable to generate a potential well in the firstsemiconductor region in conjunction with an insulator layer and collectcharges generated by the first wavelength of infrared radiation. Thelayers of semiconductor material may be varied to enhance theperformance of the resulting infrared device.

U.S. Pat. No. 5,583,338, to Goodwin, incorporated herein by reference,discloses a HgCdTe S-I-S (semiconductor-insulator-semiconductor) twocolor infrared detector wherein the semiconductor regions are groupII-VI, with different compositions for the desired spectral regions. Thedevice is operated as a simple integrating MIS device with respect toone semiconductor.

U.S. Pat. No. 5,559,336 to Kosai et al., incorporated herein byreference, discloses a radiation detector pixel unit cell with an-p+LWIR photodiode that is vertically integrated with a p+n MWIRdetector in a n-p+-n structure. Electrical contact is made separately toeach of these layers in order to simultaneously detect both the LWIR andMWIR bands. The electrical contact is made via indium bumpinterconnections so that the detector unit cell can be subsequentlyhybridized with a topside mounted electronic readout integrated circuit.The n-p+-n structure in a given pixel of an array of radiation detectorpixels is electrically isolated from all neighboring pixels by a trenchthat is etched into an underlying substrate.

U.S. Pat. No. 5,120,960 to Halvis, incorporated herein by reference,discloses an infrared (IR) imaging device with substantially identicaltop and bottom IR detector arrays. In separate embodiments, either a topor bottom surface of the top array is stacked onto the bottom array toconfront a top surface of the bottom array. Individual detector elementsand subarrays of the top array are aligned with corresponding detectorelements and subarrays of the bottom array. The image readout circuitsof both the top and bottom array are connected by wire bonding toreadout control circuits formed in the peripheral region of the wafer inwhich the bottom array is formed. These IR detector arrays are formed onseparate substrates. Furthermore, the two sub-arrays are aligned in aone-to-one pixel relationship, the purpose being to eliminatenon-operational pixels from the combined dual array.

DISCLOSURE OF THE INVENTION

The above and other objects are preferably achieved by a focal planeimaging array having a dynamic range, comprising: a detector with alarge pixel array having a plurality of large pixels, each of the largepixels having a large pixel area and a large pixel signal contact, tocreate a first signal that travels to the large pixel signal contact,and a small pixel array having a plurality of small pixels, each of thesmall pixels having a small pixel area and a small pixel signal contact,to create a second signal that travels to the small pixel signalcontacts, wherein the plurality of small pixels is greater than theplurality of large pixels, wherein the large pixel array and the smallpixel array are aligned and vertically stacked on a monolithicsemiconductor substrate; a readout integrated circuit operablyinterconnected to the large pixel signal contacts and the small pixelsignal contacts; and a clock operably connected to the large pixelsignal contacts and the small pixel signal contacts to read the firstsignals at a first clock rate having a first integration time and afirst reset time, and to read the second signals at a second clock ratehaving a second integration time and a second reset time. The firstclock rate is preferably faster than the second clock rate, and thereading of the first signals at the first clock rate, and the secondsignals at the second clock rate, extends the dynamic range of the focalplane imaging array.

The presently preferred embodiment of the focal plane imaging array alsocontains a large pixel area that is at least twice as large as the smallpixel area. The first clock rate is also preferably at least 1500 framesper second, whereas the second clock rate is preferably no more than 500frames per second.

The focal plane imaging array also preferably contains a wide band gapbarrier between the large pixel array and the small pixel array toprevent crosstalk between the large pixels and the small pixels. Thelarge pixels are responsive to a first brightness range, and the smallpixels are responsive to a second brightness range, wherein the firstbrightness range is different from the second brightness range. Thelarge pixels also absorb a first wavelength and said small pixels absorba second wavelength.

The readout integrated circuit of the presently preferred embodiment ofthe focal plane imaging array described above further preferablycomprises: a large pixel amplifier having an electrical gain forreceiving the first signal and creating a large pixel output voltage; alarge pixel comparator to receive the large pixel output voltage; aplurality of small pixel amplifiers for receiving the second signal andcreating small pixel output voltages; and a small pixel comparator toreceive the small pixel output voltages.

When the large pixel output voltage indicates that the large pixel issaturated, the large pixel comparator outputs a TRUE value that changesthe first clock rate to have a first changed integration time and afirst changed reset time, wherein the first changed integration time isshorter than the first integration time and the first changed reset timeis greater than the first reset time. If the large pixel output voltageindicates that the large pixel is unsaturated, the large pixelcomparator outputs a FALSE value and the first clock rate maintains thefirst integration time and the first reset time.

When the small pixel output voltages indicate that a preset number ofsmall pixels are saturated, the small pixel comparator outputs a TRUEvalue that changes the second clock rate to have a second changedintegration time and a second changed reset time, wherein the secondchanged integration time is less than the second integration time andthe second changed reset time is greater than the second reset time. Ifthe small pixel output voltages indicate that the small pixels areunsaturated, the small pixel comparator outputs a FALSE value and thesecond clock rate maintains the second integration time and the secondreset time.

Further, if the small pixel output voltages indicate that a presetnumber of the small pixels are saturated while using the second changedintegration time and a second changed reset time, the small pixelcomparator will output another TRUE value that changes the second clockrate so that it has a second re-changed integration time and a secondre-changed reset time, wherein the second re-changed integration time isshorter than the second changed integration time and the secondre-changed reset time is longer than the second changed reset time. Ifthe small pixel output voltages indicate that the small pixels areunsaturated while using the second changed integration time and a secondchanged reset time, the small pixel comparator will output a FALSE valueand the second clock rate will remain at the second changed integrationtime and the second changed reset time.

Ultimately, the reading of the first signals at the first changedintegration time, and the second signals at the second changedintegration time and the second re-changed integration time, extends thedynamic range of the focal plane array.

The readout integrated circuit of the focal plane imaging array alsopreferably contains a master frame reset clock to send a small pixelreset signal, which causes the second changed integration time andsecond changed reset time to return to the second integration time andthe second reset time, and causes the second re-changed integration timeand the second re-changed reset time to return to the second integrationtime and the second reset time.

The second reset time, the second change reset time, and/or the secondre-changed reset time described above also preferably causes a largepixel reset signal which changes the first changed integration time andthe first changed reset time to the first integration time and firstreset time.

The large pixel amplifier of the readout integrated circuit furthercomprises: a first integration capacitor for storing the first signaland producing the large pixel output voltage; and a second integrationcapacitor for storing the first signal. If the large pixel outputvoltage saturates, the second integration capacitor is turned on toreduce the electrical gain, and if the large pixel output voltage isunsaturated, the second integration capacitor is turned off.

The invention also preferably comprises a capacitor averaging comparatorfor comparing a plurality of pixel electrical signals, each having asignal voltage, comprising: a plurality of signal switches, each signalswitch corresponding to one of the pixel electrical signals; a bufferamplifier connected to the signal switches; a first capacitor; a firstcapacitor switch connected between the buffer amplifier and the firstcapacitor; a second capacitor; a second capacitor switch connectedbetween the first capacitor and the second capacitor; and a latchingcomparator connected to the second capacitor controlled by a comparatorcontrol voltage.

When the signal switches are sequentially closed, each of the pixelelectrical signals is sequentially sent to the buffer amplifier. Whenthe first capacitor switch is closed, the buffer amplifier drives thefirst capacitor to store a first stored voltage that is proportional tothe signal voltage. When the first capacitor switch opens, the secondcapacitor switch closes and charge is transferred between the first andsecond capacitors, causing any prior stored voltage on the secondcapacitor and the first stored voltage on the first capacitor to form aweighted average voltage. The comparator compares the weighted averagevoltage to the comparator control voltage. The weighted average is basedon the ratio of capacitance between the first capacitor and the secondcapacitor, and if any of the weighted average voltages exceeds thecomparator control voltage, then the comparator latches to a high value.

The small pixel comparator of the readout integrated circuit (describedabove) may also preferably be a capacitor averaging comparator forcomparing a plurality of pixel electrical signals, each having a signalvoltage, further comprising: a plurality of signal switches, each signalswitch corresponding to one of the pixel electrical signals; a bufferamplifier connected to the signal switches; a first capacitor; a firstcapacitor switch connected between the buffer amplifier and the firstcapacitor; a second capacitor; a second capacitor switch connectedbetween the first capacitor and the second capacitor; and a latchingcomparator connected to the second capacitor controlled by a comparatorcontrol voltage.

When the signal switches are sequentially closed, each of the pixelelectrical signals is sequentially sent to the buffer amplifier. Whenthe first capacitor switch is closed, the buffer amplifier drives thefirst capacitor to store a first stored voltage that is proportional tothe signal voltage. When the first capacitor switch opens, the secondcapacitor switch closes and charge is transferred between the first andsecond capacitors, causing any prior stored voltage on the secondcapacitor and the first stored voltage on the first capacitor to form aweighted average voltage. The comparator compares the weighted averagevoltage to the comparator control voltage. The weighted average is basedon the ratio of capacitance between the first capacitor and the secondcapacitor, and if any of the weighted average voltages exceeds thecomparator control voltage, then the comparator latches to a high value.

The above and other objects are also preferably achieved by a processfor using a focal plane imaging array having a background brightness anda detector operably connected to a readout circuit having a large pixelarray with a plurality of large pixels, each of the large pixels havinga large pixel area and a large pixel signal contact, to absorb a firstwavelength, and a small pixel array having a plurality of small pixels,each of the small pixels having a small pixel area and a small pixelsignal contact, to absorb a second wavelength, wherein said plurality ofsmall pixels is greater than said plurality of large pixels, and whereinthe large pixel array and the small pixel array are aligned andvertically stacked on a monolithic semiconductor substrate, comprising:illuminating the large pixel array and small pixel array with aninfrared light source to create a first signal that travels to the largepixel signal contacts, and a second signal that travels to the smallpixel signal contacts; clocking the first signal at a first clock ratehaving a first integration time and a first reset time, and the secondsignal at a second clock rate having a second integration time and asecond reset time, wherein the first clock rate is faster than thesecond clock rate; and shortening the first integration time if thefirst signal saturates the large pixels between the first rest times,and reducing the second integration time if the second signal saturatesthe small pixels between the second reset times. This process achievesan extended dynamic range and better clutter rejection for the focalplane array.

The process also preferably includes determining an average for thebackground brightness; processing the first signal to detect fast eventshaving a fast event brightness that occurs within a time range, and thesecond signal to detect standard events having a standard eventbrightness, wherein the fast event brightness is comprised of thebackground brightness and an actual fast event brightness, and thestandard event brightness is comprised of the background brightness andan actual standard event brightness; calculating the actual fast eventbrightness and the actual standard event brightness; comparing theactual fast event brightness and the actual standard event brightness toapproximate a temperature range for the fast events and the standardevents; and determining whether the fast events and the standard eventsare threatening based on the temperature range.

The step of calculating the actual fast event brightness and the actualstandard event brightness preferably comprises: determining a spatialcorrection factor for the fast event brightness based on the spatialextent of the standard event brightness, and a temporal correctionfactor for the standard event brightness based on the duration of thefast event brightness; correcting the background brightness using thespatial correction factor to achieve a spatially corrected backgroundbrightness, and using the temporal correction factor to achieve atemporally corrected background brightness; and subtracting thespatially corrected background brightness from the fast event brightnessto achieve the actual fast event brightness, and subtracting thetemporally corrected background brightness from the standard eventbrightness to achieve the actual fast event brightness.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1—Dual Pixel Pitch Array, Concept

FIG. 2—Dual Pixel Pitch Array, Top Plan View

FIG. 3—Etched Planar Structure, Top Plan View

FIG. 4—Etched Planar Structure, Side Elevational View

FIG. 5—Large Pixel with Windows for Small Pixels

FIG. 6—Etched Planar Structure, Side Elevational View, Alternative A

FIG. 7—Etched Planar Structure, Side Elevational View, Alternative B

FIG. 8—Mesa Structure, Side Elevational View

FIG. 9—Mesa Structure, Top Plan View

FIG. 10—Back-to-Back Two-Color Photodiode Arrangement, Side ElevationalView

FIG. 11—Back-to-Back Photodiodes as a Dual Pixel Pitch Array, SideElevational View

FIG. 12—Pixel Time Signals

FIG. 13—Large Pixel Timing Logic States and Transition Events

FIG. 14—Small Pixel Timing Logic States and Transition Events

FIG. 15—Circuit Elements for Extended Dynamic Range Operation

FIG. 16—Averaging Comparator Circuit

FIG. 17—Preferred Large Pixel Amplifier Circuit Block

FIG. 18—Normalizing for Spatial Extent to Support Two-Color RatioCalculations

FIG. 19—Normalizing for Temporal Extent to Support Two-Color RatioCalculations

DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENT

The presently preferred embodiment of the invention is a monolithic,two-color, dual pixel pitch focal plane array with variablecomplementary timing and an extended (increased) dynamic range.

Infrared (IR) focal plane arrays consist of two parts: a detector and areadout integrated circuit (ROIC). These two parts are interconnectedusing well-known techniques such as hybridization with indium bumpconnections. The detector and readout combination is referred togenerically as a sensor chip assembly (SCA). The presently preferredembodiment of the invention uses these well-known methods.

The detector of the present invention is preferably monolithic, that is,formed in a single piece. It preferably contains two arrays, preferablyoperating in two colors (so the sensing elements for one color do notblock the sensing elements for the other, as described below),preferably aligned and vertically stacked on a monolithic (single)semiconductor substrate (structure). It is made using well-knowntechniques for growing and processing multi-layered detectors. P-on-ndiodes are described in the detailed descriptions below, but n-on-p mayalso be used, if applicable. The two colors can be any two IR bandswithin the sensitivity range of the materials that might be used. Thepresently preferred invention can be implemented in all commonly usedquantum detectors, for example made of HgCdTe, InAs—GaSb strained layersuper-lattice, etc.

The present invention preferably uses arrays that are structured so thatthey function as one being on top of the other, as depicted in FIG. 1with arrays 101 and 102. Because one array sits on top of the other, thelight 104 for the second array 102 passes through the first array 101.This automatically results in two-band operation, since the first array101 absorbs one set of colors, shown as light 103, while allowing othercolors 104 to pass through to the second array 102.

The first major difference between the presently preferred embodiment ofthe invention and prior art two-color IR arrays is that one of the twoarrays in the present invention is preferably comprised of much largerpixels than the other. The larger pixels are preferably made at aspecific integer multiple of the smaller, standard sized, pixels. Whileany ratio of 2 times larger or greater could be used, the preferreddesign is for substantially oversized pixels that are approximately 10×,25×, or even 100× larger in area. This is depicted in FIG. 1, in whichlarge pixels 105 in array 101 have an area that is 9× larger than thesmall pixels 106 in array 102.

At first glance, making one array with such substantially oversizedpixels seems to be odd and undesirable. Very significant research anddevelopment resources have been expended to develop high pixel countarrays with very small pixels, which increases resolution significantly.The large pixels according to the present invention sacrifice resolutionin one color. Furthermore, because they are so large, they potentiallycause electrical design problems (e.g. larger transistors may benecessary, different bias settings could be required, and different gaindesigns could be needed). In fact, for some popular two-color designs inuse today, this arrangement will not work.

However, the large/small pixel combination has many advantages thatallow the presently preferred embodiment of the invention to betteraddress the challenges for military/security sensors than traditionalsensors. Many of these advantages may not be immediately obvious andothers are realized through a combination of features. To fully realizethese advantages requires a combination of (1) different clocking of thetwo separate arrays; (2) methods for extending (increasing) the dynamicrange of the sensor utilizing variable clocking; and (3) new methods fortwo-color radiometry. However, any of these methods implemented singly,rather than in combination, still offers certain advantages.

In the presently preferred embodiment of the invention, the two separatearrays within the invention are clocked at two different rates, one fastand one slow. This fast/slow clocking addresses the military/securityneed for sensing of very fast events while simultaneously performingslower, staring sensing. It also mimics human visual perception, whichincludes fast, involuntary sensing along with “normal” scanning of ascene to see detail. In the presently preferred embodiment of theinvention, the fast array is preferably clocked from 1500 to 5000 framesper second; alternate embodiments can include clocking at a faster orslower rate than this preferred range. The small pixel array is clockedat a much slower speed. This preferably ranges from 30 frames per secondto as fast as 500 frames per second, depending on the needs of theapplication.

The large pixels are clocked fast even though this presents obviouselectrical design challenges. The large pixels have larger capacitance(i.e. larger ability to store an electrical charge), and therefore,their electronic amplifiers require much larger current drive capabilitythan would be required for the smaller pixels if they were clocked fast.However, clocking the small pixels at rates of 1500 to 5000 frames persecond results in an output data rate that is far too high. For example,an array in the popular video graphics array (VGA) IR format of 512×640pixels would produce an output data rate that is over 5× faster thanavailable output and processing technologies can support. But becausethere are fewer of the larger pixels— 1/10^(th), 1/25^(th), or even1/100^(th) of the number of small pixels—they can be clocked at thesehigh rates without producing excessive data output rates.

The present invention achieves very high dynamic range operation bycombining two methods. The first utilizes offset optical responsivitiesfor the large and small pixels such that they respond to two differentlight brightness ranges. For example, large pixels can be designed andoperated to respond to optical flux levels ranging from 10⁹ to 10¹³photons/sec/cm2, whereas small pixels can be designed and operated torespond to optical fluxes ranging from 10⁶ to 10¹⁰ photon/sec/cm2. Inthis case, the large, fast-clocked pixels will not saturate with verybright signals, whereas the small pixels will. Conversely, the highersensitivity of the small pixels enables very dim objects to be detected,whereas these objects will be below the noise threshold for the lowersensitivity large pixels. In this example, the combined operation of thetwo pixel arrays provides a dynamic range of 10 million, from 10⁶ to10¹³ photons per second.

Additional dynamic range improvement may be realized through variableintegration timing, in which standard integration time (the time duringwhich the pixel amplifier circuit integrates generated photocurrent fromthe detector) is dynamically switched to a shorter time. This allows forextended dynamic range operation in both colors individually rather thanrealizing it through the combination of the two colors as above. Thisallows for more robust sensing and the use of two-color radiometry forthe entire extended range of operation. These and other techniquesprovide a dynamic range in excess of 150 dB for the present invention toachieve non-saturated operation for radiometric characterization ofextremely bright events.

A non-obvious advantage of the present invention's architecture isimproved clutter rejection. The presently preferred embodiment combinestwo-color radiometric techniques with multi-scale spatial and temporaldata to provide the equivalent of three independent data streams forclutter rejection: two-color radiometry, dual timescale data, and dualspatial scale data. The multiple data streams allow correspondingmultiple event filters to be applied to the problem of clutter and falsealarm rejection. This will improve decision accuracy through the use ofmultiple Bayesian probabilistic tests or similar methods. For example, abright momentary spot seen on the slow clocking small pixels that doesnot change on the fast clocking large pixels could be rejected as notbeing a fast event, such as gunfire or rocket propelled grenade launch.

A final non-obvious advantage of the present invention is improveddetection performance in airborne environments. In these environments,air turbulence causes optical lensing effects that spread energy over awider area on any focal plane array. In a typical sensor, this reducesdetection performance because multiple photodiodes will contributenoise, thus lowering signal-to-noise. The large/small pixel combinationin the present invention allows for a dual detection approach: standardmethods with the small pixels, and detection with a larger spatialextent with the large pixels. Because the large pixels provide anindependent signal that is less susceptible to turbulent air opticallensing effects, the detection according to the present invention ismore robust.

Alternate implementations also offer advantages. Combining dual formatpixels with dual clocks, one fast clock for large pixels and one slowclock for small pixels, adds considerable advantages. Thisimplementation, although simplified from the presently preferredversion, realizes the full advantages of spatial-temporal clutterrejection. A fast clock rate of 1500 frames per second or greaterenables sensing the full range of threats consistent with military andsecurity needs. In this alternate implementation, dynamic rangeimprovements of 20 to 60 dB are practical through a combination ofdesign and operational settings to enable offset responsivities for thelarge and small arrays.

Adding dynamically switched integration times for the two colors extendsthe dynamic range for both colors, enabling two-color radiometry andmore robust sensing for the entire sensing range.

Two-color radiometry, enhanced detection, and improved clutter rejectionfor any of these combinations can be realized flexibly through off-chipprocessing.

The presently preferred embodiment of the invention can be furtherdescribed in FIGS. 1-19. The detector of the present invention combines,in one monolithic structure, two aligned and vertically stacked pixelarrays, one with large pixels, and the other with smaller pixels. Eachpixel in the large pixel pitch array will be aligned with a sub-array ofthe small pixel pitch array. Small pixel sub-arrays (i.e. the number ofsmall pixels inside the boundary of a large pixel) may be of any size,from a two by two array to larger arrays of N×M pixels, where N and Mare greater than two. The smaller pixels will be square in shape, butthe larger pixels could be square or rectangular. There will be M timesN fewer large pixels than small pixels. These two arrays will becontrolled with two separate clock signals, and these signals will bedynamically switchable to enable extended dynamic range operation.

Infrared (IR) light is electromagnetic radiation with longer wavelengthsthan those of visible light. It extends from the nominal red edge of thevisible spectrum at 0.74 μm to 300 μm. The wavelengths absorbed by thepresent invention can be adjusted. For example, wavelength 103 (shown inFIG. 1) could be Short Wave IR (SW) (between 1.5 μm-2.5 μm) andwavelength 104 (shown in FIG. 1) could be Mid-Wave IR (MW) (between 2.5μm-5.0 μm). This would comprise a SW/MW detector. Similarly, 103 couldbe MW and 104 could be Long Wave IR (LW) (between 7 μm-12 μm), whichwould comprise a MW/LW detector. It is also possible for 103 to be ashorter MW wavelength, such as the 3.4 μm to 4.2 μm band, and for 104 tobe a longer MW wavelength, such as the 4.4 μm to 5 μm band. This wouldcomprise a MW/MW detector.

FIGS. 3 through 11 depict physical structures that can be used toimplement the detector of the present invention. FIGS. 3 through 9 showpreferred implementations and FIGS. 10 and 11 show alternateimplementations. Any of these specific methods may be used to implementthe large/small pixel structure. In addition, methods derived from thesebasic approaches may also be used.

FIG. 2 shows a corner of a large array with large pixel 201 and smallpixels 202. In FIG. 2, there are 5×5 small pixels for each large pixel.Small pixels are shown with signal contact 203. There are no othercontacts or grounding structures shown in FIG. 2 for clarity. Thesedetails, which differ for different detector structures, are depicted inFIGS. 3 through 9.

FIGS. 3 and 4 depict the planar, etched structure, showing a top planview in FIG. 3 and a side elevational view in FIG. 4. These structuresare fabricated using well-known techniques (Jagmohan Bajaj et. al.,“Comparison of Type-II Superlattice and HgCdTe Infrared DetectorTechnologies,” Infrared Technology and Applications XXXIV, edited byBjørn F. Andresen, Gabor F. Fulop, Paul R. Norton, Proc. of SPIE Vol.6542, 65420B (2007), incorporated herein by reference). Etched planarstructures are a class of detector geometries characterized by growinglayers on a substrate, and then performing etch and implantation stepsto form diode structures. This differs from mesa structures in whichdiode structures are formed with etch steps only.

FIG. 4 shows layers 401, which is a band 1 n-type (or p-type) material,402, which is a wide bandgap isolation layer, 403, which is a band 2n-type (or p-type) material, and 404, which is a p-type (or n-type)material. These layers are grown on a substrate using well-knownmethods. The substrate is not shown in FIG. 4 for clarity. The grownlayers are then etched and processed with implanted material to form thelarge and small pixels of the present invention. FIG. 4 shows how lightof two different wavelengths, 103 and 104, are absorbed near the p-njunctions of the large and small pixels.

Small pixel 202 (shown in FIG. 2) consists of elements 301, 302 and 303(shown in FIGS. 3 and 4). 301 is an n-type (or p-type) material that ispart of layer 401, which is exposed through an etch step to make each ofthe small pixels. 302 is an implanted p-type (or n-type) material.Together, 301 and 302 form the p-n junction for the small pixelphotodiode. 303 is a contact material that is deposited on top of 302.303 is the signal contact for the small pixel. It is connected to thereadout circuit using well-known methods. In the etched planarstructure, the 5×5 array of small pixels are grounded with groundconnect 304 (also shown in FIG. 4). The ground connect can beimplemented in various shapes and sizes. The ground connect in FIG. 4 isshown as a connection along the edge of large pixel 201.

Large pixel 201(shown in FIG. 2) consists of elements 305, 306, 307 and308, all of which are shown in FIGS. 3 and 4. Element 306, the n-type(or p-) material, is almost completely covered by 305, the p-type (orn-) material. Together, 305 and 306 form the p-n junction of the largepixel photodiode. From the top plan view in FIG. 3, 306 is seen alongthe edge of large pixel 201, near ground contact 304 and indicated bythe single-hatched area. In FIG. 3, element 305 is indicated by thecrosshatched area.

Note that for the detector shown in FIGS. 3 and 4, the small pixels 202form holes or windows within the boundary of the large pixel 201. FIG. 5shows the large pixel 201 as the hatched square area with small pixels202. The surface of the etched planar large pixel as shown in FIGS. 3and 4 is like a waffle, with holes for small pixels and a single largepixel defined by a raised contiguous area. Although overlaying the smallpixel area 301 as indicated in FIG. 4, the large pixel does not blockthe small pixel because the large and small pixels are preferablysensitive to different colors.

FIGS. 3 and 4 also show signal contacts 307 and ground contacts 308 forthe large pixel. These contacts can be of any size or shape. Because thelarge pixel has openings in it, several signal contacts 307 are usedrather than a single large contact. Both 307 and 308 are depicted asmaterial that is deposited onto the detector structure. These elementscan also be formed by growing and etching additional layers.

The 5×5 array depicted in FIGS. 3, 4 and 5 is only one possibleimplementation of the present invention. Small pixel sub-arrays may beof any size from 2×2 or larger. Preferably, small pixels are separatedfrom large pixels by a wide bandgap barrier layer 402 (shown in FIG.4)—this is a preferred implementation, but may not be used in otheralternate implementations. The barrier isolates the two overlapped pixelstructures to avoid crosstalk between pixels.

The separate ground contacts ensure high-quality grounds to the readoutwith minimized crosstalk for both large and small pixel arrays. Thearrangement of ground contacts 304 and 308 (shown in FIG. 4) is one ofmany possible arrangements. These ground contacts may be separatelyconnected to ground paths in the readout circuit, or they may beconnected as a single ground.

FIGS. 3 and 4 depict a planar etched structure that implements thepresent invention where the band 1 pixel is the small pixel and the band2 pixel is the large pixel. Band 1 absorbs shorter wavelength light, andband 2 absorbs longer wavelength light.

This arrangement can be reversed as shown in FIGS. 6 and 7. In FIG. 6,the same four-layer structure is used. Small pixels are formed by firstetching trenches to delineate them. Elements 301, 302 and 303 form thesmall pixels. Ground contact 304 is positioned differently than in FIG.4, and a different etch depth is required. The large pixel is formed byelements 305, 306, 307, and 308. Large pixels are delineated by an etchinto layer 401; this etch is not shown in the figure for clarity. Itdivides layer 401 into multiple elements 306, rather than the singleelement 306 that is shown. Within the single large pixel boundary,multiple elements 305 are used to make the p-n junction for the largepixel photodiode. These dispersed junctions avoid the need to eliminateone or more small pixels, which is the case for the example of FIG. 7.Here, in FIG. 6, all of the contacts 307 connect to a single unit cellamplifier for the large pixels. These contacts are similar to the smallpixel contacts of FIG. 4, but they are spaced differently and there arefewer of them.

FIG. 7 depicts another method for implementing a detector according tothe present invention such that the small pixels absorb the longerwavelength radiation and the large pixel absorbs the shorter, which isthe opposite of the methods depicted in FIG. 4. In FIG. 7, the detectormaterial is formed by growing only three layers on the substrate. Layer701 is n-type (or p-) material, layer 702 is a wide bandgap isolationlayer, and layer 703 is made of n-type (or p-) material, just like layer701. Small pixels are formed by first etching trenches through 703 andinto 702 to form 301. 302 is formed by implanting p-type (or n-)material into 703 (301). This forms the p-n junction for the small pixelphotodiode. 303 is contact material that is deposited onto 302; itprovides a signal contact to the readout circuit. A four-layer versionof this band 1/band 2 structure can be made with a p-type (or n-)material grown on top of 703 and etching trenches. The grown layer thenforms 302 rather than implanted material. A five-layer version is alsopossible, in which a contact material layer is grown and elements 304and 303 are formed by etching this layer. In any of these cases,elements 304 provide a ground contact for the group of small pixels 202that are within the boundary of the single large pixel 201.

In FIG. 7, large pixels are formed by etching a large window 704 intoand through layers 703 and 702, then implanting p-type (or n-) materialinto 701 (306) to form 305. Together 305 and 306 form the p-n junction.Contact material 307 is deposited onto 305. Elements 308 provide groundcontact. 307 provides a signal contact to the readout circuit. The largewindow requires that one or more small pixels are eliminated, but inthis structure, as few as only three layers are required. This ispotentially an advantage since growing a smaller number of layers isfaster and less expensive.

As FIGS. 4, 6 and 7 show, the etched planar structure allows for thelarge pixel to be implemented in either band 1 or band 2 material. Forthe arrangements shown in FIGS. 6 and 7, the large pixel is continuous,with no windows as depicted in FIG. 5; however, the small pixel fillfactor may be reduced compared to the FIG. 4 arrangement. In FIG. 4, theeffective quantum efficiency of the large pixel is reduced because it isnot a solid square.

FIGS. 8 and 9 depict a vertical mesa structure detector according to thepresent invention. These are preferably fabricated using well-knowntechniques (Michel Zécri, Patrick Maillart, “Advanced ROICs Design forCooled IR Detectors”, Infrared Technology and Applications XXXIV, editedby Bjørn F. Andresen, Gabor F. Fulop, Paul R. Norton, Proc. of SPIE Vol.6940, 69402×, (2008), incorporated herein by reference, and C. H. Grein,P. Boieriu, M. E. Flatté, “Single- and two-color HgTe/CdTe-superlatticebased infrared detectors”, Quantum Sensing and Nanophotonic Devices III,edited by Manijeh Razeghi, Gail Brown, Proc. Of SPIE Vol. 6127,incorporated herein by reference). The vertical mesa structurerepresents a class of detectors in which no implantation methods areused. All required elements are formed by growing layers and etching.These detectors typically have more layers than planar etched detectors.

The vertical mesa structure is first grown as layers on a substrate. InFIG. 8, these include layers 801, which is a band 1 n-type (or p-)material, 802, which is a p-type (or n) material, 803, which is a widebandgap barrier layer for isolation, 804, which is a band 2 n-type (orp-) material, and 805, which is a p-type (or n-) material. These layersare etched to form the small and large pixels, including etching thetrenches 806 for small pixels and etching window 807, which is used forlarge pixel ground connection 308, and which also delineates the largepixel. Small pixels consist of elements 301, 302, and 303. Elements 301and 302 form the p-n junction to make the small pixel photodiodes.Element 303 is the small pixel signal connection. It consists of specialcontact material, and it connects to the readout circuit usingwell-known methods. Ground connection 304 is used for all of the smallpixels within the boundary of the single large pixel. The large pixelconsists of elements 305, 306, 307 and 308. Elements 305 and 306 formthe p-n junction to make the large pixel photodiode. Element 307 isconnection material. It connects to the readout circuit using well-knownmethods. In this embodiment of the invention, there are 7 rows and 7columns of small pixels within each large pixel boundary. One smallpixel is lost to make room for connection 307.

FIG. 9 shows the same vertical mesa structure of the present inventionfrom a top plan view. Not all elements are visible or fully visible inthis view for clarity. Small pixel elements 302 and 303 are shown;element 301 is hidden by 302. Ground contacts 304 are shown as cornercontacts; this is one of many possible shapes and positions for thesecontacts. For the large pixel, elements 307 and 308 are shown. Theground contacts 308 can be realized in other shapes and positioned inother areas, if required or desired. Element 901 is a partial view oflarge pixel element 305, which is part of layer 802. Layers 801 and 802can be seen on the edge of the large pixel in this top down view. Layer804 can be seen in the trenches between small pixels.

For the vertical mesa structure (depicted in FIGS. 8 and 9), the largepixel functions as a single square pixel with no windows for smallpixels, as shown in FIG. 5 for the planar detector.

The so-called “back-to-back” triple heterojunction structures shown inFIGS. 10 and 11 require a special approach to implement the presentinvention's architecture. These two-color focal plane arrays (FPAs) aremade using well-known methods (E. P. G. Smith et al, “Status ofTwo-Color and Large Format HgCdTe FPA Technology at Raytheon VisionSystems”, Quantum Sensing and Nanophotonic Devices III, edited byManijeh Razeghi, Gail Brown, Proc. Of SPIE Vol. 6127, incorporatedherein by reference, and P. R. Bratt et al, “Historical Perspective onHgCdTe Material and Device Development at Raytheon Vision Systems,”Infrared Technology and Applications XXXIV, edited by Bjørn F. Andresenet al, Proc. Of SPIE Vol. 6542, 6542B, (2007), incorporated herein byreference).

FIG. 10 depicts the back-to-back device structure. Three layers arepreferably grown on a substrate: 1001 is n-type (or p-) material; 1002is p-type (or n-) material; 1003 is n-type (or p-). Each pixel isdelineated with an etched trench 1004. Ground contact 1009 is used forboth band 1 and band 2 as is signal contact 1005. In this structure, thebias is preferably switched rapidly so that at one moment in time band 2n-type 1006 and p-type 1007 form a reversed bias photodiode while band 1n-type 1008 and p-type 1007 form a forward biased diode. Immediately,the bias is reversed so that band 1 n-type 1008 and p-type 1007 form areversed bias photodiode while band 2 n-type 1006 and p-type 1007 form aforward biased diode. Band 1 light 103 is absorbed only when 1008 and1007 form a reversed biased p-n junction. Band 2 light 104 is absorbedonly when 1006 and 1007 form a reversed biased p-n junction. Wheneverthe bias is switched, signal switch 1010 switches the photo-generatedsignal so that it connects to the unit cell readout 1011 for band 1 orcell readout 1012 for band 2, and thus the two bands generateindependent signals.

FIG. 11 depicts how to create the present invention in the back-to-backtriple heterojunction architecture. Either band 1 (1011) or band 2(1012) signals are aggregated by means of common connection 1101 on oneof the switch 1010 positions. In the figure, connections 1012 for band 2are aggregated in this manner. From a device standpoint, this is similarto the multiple contact arrangement of FIG. 6. However, signals are notas well isolated, and so, this is an alternative implementation.

In preferred implementations, the detector of the present inventionspecifically provides for separate ground paths for small and largepixels. The ground contacts 304 and 308 (shown in FIGS. 3, 4, and 6through 9) are physically separated to minimize coupling. This designhelps avoid crosstalk between the small and large pixels and helpsensure that high-speed clock signals do not cause excess noise in thesmall pixel signals. Also, in the preferred implementation, the smallpixel layer is separated from the large pixel layer by a wide bandgapisolation layer, which is shown variously in the different preferreddetector structures as 402 (FIGS. 4 and 6), 702 (FIG. 7) and 803 (FIG.8).

FIG. 12 illustrates the clock signals that are used preferentially inthe present invention's readout integrated circuit (ROIC). There arethree types of clock signals: the master frame reset clock, labeled asφR; the small pixel clock, labeled as φS and consisting of φS1, φS2, andφS3; and the large pixel clock, labeled as φL and consisting of φL1 andφL2. The master reset clock signals operate across the entire array tocontrol both large and small pixel arrays of the present invention.Pixel clock signals φS1, φS2 , φS3, φL1 and φL2 are used simultaneouslyacross the array, with different pixels or groups of pixels beingcontrolled by the different clock signals from this group.

In alternative implementations, a reset pulse signal may be used ratherthan the reset clock signal φR, the difference being that the resetpulse signal is sent asynchronously rather than at regular intervals. Inthis case, the reset signal is generated by external logic. All otheraspects of the operation of the clock signals remain unchanged if anaperiodic reset pulse is used instead of a reset clock signal.

FIG. 12 shows these clock signals, and their timing relationships. InFIG. 12, one master reset clock cycle is shown, indicated by time 1200.Time 1200 is much longer than the small pixel frame time, indicated bytime 1201. Three small pixel frames are shown at a magnified scale, asindicated by time marker 1220, which is the start of the master frametime and the start of the first small pixel frame, and by time marker1221, which indicates a short portion of the entire time period 1200. Asindicated by 1221, the entire master reset time period 1200 is equal toa large number of small pixel frame times 1201, such as 50 or 100. Foreach master reset, the small pixel clock cycles through this set numberof frames. In the case of an aperiodic reset signal, the small pixelclock will cycle through a variable number of frames. Time markers 1222and 1223 show a large number of the large pixel frame times 1211 arewithin a single small pixel frame time 1201. Just as for the time 1201to time 1200 relationship, time 1201 is much longer than time 1211.Typically, each small pixel frame time 1201 equals 10 to 20 large pixelframe times 1211, and for each small pixel frame, the large pixel clockwill cycle through this set number of large pixel frames. As timemarkers 1220 and 1222 indicate, the clock signals are synchronized suchthat when the master frame reset time period begins (marker 1220), so doa small pixel frame time 1201 and a large pixel frame time 1211.Similarly, whenever a small pixel frame time 1201 begins within a resettime 1200, so does a large pixel frame time 1211. For example, if thereare 20 large pixel frames 1211 per small pixel frame 1201, and 50 smallpixel frames 1201 per master reset frame 1200, there would be 1000 largepixel frames per master reset.

In FIG. 12, each frame time consists of an integration (accumulation oraddition) period and a reset time. Pixel amplifiers will integrate(accumulate or add) photo-generated current during the integration timeof their φS and φL signals. During the reset periods, pixels integrateno signal, and the prior signal values are reset to zero. For smallpixels, frame time 1201 typically consists of integration time 1202 andreset time 1203, as shown in the φS1 trace. In certain situations,shorter integration times 1204 or 1206 are used, and since the framerate does not change, the pixels are held in reset for a longer time,1205 or 1207. The clock signals for these shorter integration times areshown by the φS2 and φS3 traces. For large pixels, frame time 1211typically consists of integration time 1212 and reset 1213, except incertain situations where shorter integration time 1214 is used alongwith correspondingly longer reset time 1215. The shorter integrationtime is shown in trace φL2; the typical operating condition is shown inφL1.

An extended dynamic range of operation for both arrays is realized byusing the variable integration times indicated in FIG. 12 by traces φS2,φS3, and φL2. FIGS. 13, 14 and 15 describe how the variable integrationtiming is controlled. The timing control logic is located on the sensorchip assembly (SCA), i.e. the combined detector and readout integratedcircuit, for preferred implementations, but it or portions of it can belocated off the SCA in alternate implementations.

Small pixel timing is preferably set for groups of small pixels, notindividually. At a minimum, each group of small pixels is defined as theset within a single large pixel boundary. Alternatively, a group ofsmall pixels may be defined as the set within the boundary of severalcontiguous large pixels. Within a group, all small pixels will use thesame pixel clock signal, but for different groups, different pixel clocksignals may be used. In this discussion, we use the example of a groupof small pixels defined as the set within the boundary of a single largepixel. Therefore, common logic controls timing for the single largepixel and multiple small pixels. For purposes of discussion, FIG. 15shows connections to three small pixels. In actual implementation, therewill be N small pixels, where N is the number of small pixels connectedto the control circuit for a single large/small pixel group.

FIG. 15 depicts the ROIC and control circuits for the preferredimplementation of the present invention's large/small pixel group. Theconcept for the circuits and control logic described in FIG. 15 is thatthe output of pixel amplifiers is sensed with a voltage comparator (adevice that compares two voltages and switches its output to indicatewhich is larger) to determine if it has saturated. If a large pixelamplifier has saturated, the gain is lowered for the next frame byreducing integration time for that large pixel. The concept is similarfor small pixels, the only difference is that gain is lowered when agroup of small pixels saturates rather than a single small pixel. Thesize of the group can be controlled flexibly in preferredimplementations. For small pixels, this concept can be extended so thatintegration time is lowered in two steps rather than a single step,allowing for a greater total range of gain control. In large pixels, itis not practically possible to reduce gain in two steps by reducingintegration time twice, because this would result in an impracticallyshort integration time. Thus, for large pixels, a second gain reduction,if desired, is achieved by adding capacitance to reduce thetransimpedance amplification ratio. Once a pixel has been set to a lowergain value, it will be reset to normal values after a set time interval.This ensures that pixel gain will be set appropriately once the scene orevent returns to normal brightness levels. Logical bit values areencoded into the output to indicate the gain state of the large andsmall pixels.

In FIG. 15, when light 103 of a first color strikes the large pixeldiode 201 on the detector, electrons are produced, resulting inphotocurrent 1530. Integrating amplifier block 1501 receivesphotocurrent 1530 and produces output 1510, which is a voltage.Similarly, light 104, a second color, strikes multiple small pixeldiodes 202, resulting in photocurrents 1531, 1532 and 1533 which arereceived by integrating amplifiers 1502. Amplifiers 1502 are identicalfor each small pixel but are different from amplifier 1501. Amplifiers1502 produce output voltages 1511, 1512, and 1513, which are typicallydifferent from each other and from voltage 1510. Amplifiers 1501 and1502 can take multiple well-known forms such as CapacitiveTransimpedance Amplifier (CTIA), buffered direct input, etc.

For large pixels, voltage 1510 from integrator 1501 is input tocomparator 1503. Control voltage 1520 sets the exact operating point forcomparator 1503. Whenever voltage 1510 nears saturation, exceedingcontrol voltage 1520, the output of comparator 1503 on connection 1515will swing to a logic 1, or TRUE value. This input to logic block 1505indicates whether or not the pixel is in saturation. Logic block 1505also receives the timing clock signals φL1, φL2, and φR. It maintainsinternal state information for the large pixel, which is described byFIG. 13.

In FIG. 13, three states are indicated: state 1301, which is standardintegration timing; state 1302, which is short integration timing; andstate 1303, which is auxiliary gain reduction. State transition eventsare indicated by 1310, which is comparator 1503 outputs a TRUE value; by1320, which is small pixel clock reset; and by 1330, which is masterframe reset.

In typical operating conditions, large pixel amplifier 1501 (shown inFIG. 15) does not saturate, and comparator 1503 voltage output on 1515remains at logical FALSE. In this case, the large pixel remains in state1301 (shown in FIG. 13) and the clock output, φL, is φL1 as shown inFIG. 12, which is the standard timing. If the pixel approachessaturation, comparator 1503 will output TRUE on 1515. The comparator isan analog element, but its input is evaluated at the end of φLintegrate, 1212 (shown in FIG. 12). Comparator 1503 TRUE output is statetransition 1310, so logic 1505 enters state 1302. This causes thefollowing events:

-   -   The clock output is switched to φL2, which is valid on the next        φL frame; and    -   Logic signal bit b0 swings to a TRUE value and logic bit b1        remains at a FALSE value; these values will be encoded into the        output stream to indicate the gain setting for the large pixel.

Logic 1505 will remain in state 1302 until one of the three statetransition events 1310, 1320 or 1330 occurs. If either 1320, small pixelreset, or 1330, master frame reset, occurs, logic 1505 transitions backto state 1301, which is standard integration timing. In this case, clockoutput is switched to φL1 , standard integration timing, which is validon the next φL frame, and bit b0 swings to a FALSE value. If event 1310(comparator 1503 TRUE) occurs from state 1302, logic 1505 transitions tostate 1303, which is auxiliary gain that may be implemented in someembodiments. In this case and if the auxiliary gain is implemented, bitb1 swings to a logical TRUE value, clock output is unchanged (φL2 ), andthe output of optional AND gate 1507 becomes TRUE, which is the optionalgain control signal 1517. The gain signal is used optionally inintegration block 1501 for additional gain control if this is available.

For small pixel operation, as shown in FIG. 15, logic block 1505controls clock input φS input to all the small pixel integrators 1502.Logic block 1505 receives input from comparator 1504 on connector 1516.The operation of comparator 1504 is controlled by input 1521, which is athreshold control voltage. One 1521 voltage value is used for the entirearray of the present invention. Comparator 1504 operates such thatvoltage 1516 will remain at logical FALSE unless a preset number of thesmall pixel integrators 1502 within the group controlled by 1505 are ator near saturation.

In the preferred embodiment, comparator 1504 is implemented as acapacitive averaging circuit combined with a standard latchingcomparator such that when the group average of small pixel integrators'1502 output (voltages 1511, 1512, 1513) exceeds the threshold controlvoltage 1521, the output of comparator 1504 swings to logical TRUE. FIG.16 describes the capacitive averaging circuit.

Alternate embodiments are also possible, for example, multiple standardcomparators with M out of N voting such that when a set number ofindividual comparators output logical TRUE, the combined comparator 1504will output logical TRUE.

FIG. 14 depicts the logic states and operation for logic 1505 control ofsmall pixel timing. Three states are indicated: 1401, standard timing;1402, short integration timing; and 1403, very short integration timing.State transition events are 1410, which is comparator 1504 value islogical TRUE, and event 1420, which is master frame reset.

In typical operating conditions, the pixels do not saturate, and voltage1516, the output of comparator 1504 remains at logical 0, or FALSE. Inthis case, the clock output φS is typical with integration time 1202 andreset time 1203 as shown in FIG. 12, φS1 . If the preset number ofpixels approach saturation, comparator 1504 will output a TRUE voltagelevel on connector 1516 as input to logic 1505. This is transition event1410, which causes logic 1505 to transition to state 1402, and thefollowing events occur:

-   -   Small pixel clock output φS is switched to φS2 , as depicted in        FIG. 12, effective in the next φS frame; and    -   Bit b2 value swings to TRUE; bit b3 remains at a FALSE value

Logic 1505 will remain in state 1402 until either transition event 1410.(comparator 1516 TRUE) or transition event 1420 (master frame reset). Inthe case of event 1420, logic 1505 returns to state 1401, bit b2 swingsto a logical FALSE value, and φS1 will be output as the small pixelclock signal in the next φS frame. In the case event 1410 from state1402, logic 1505 transitions to state 1403, both b1 and b0 swing tological TRUE values, and clock signal φS2 is output on the next φSframe.

The circuit elements 1503, 1504, and 1505, provide 150 dB dynamic rangefor small pixels (seven and one-half decades) when operated as explainedabove and with 14 bit ADC and typical operating rates for clocks. Otherimplementations of the present invention could eliminate the very shortintegration clock for small pixels, φS3 , which would still providedynamic range of 115 to 125 dB in typical implementations with 14-bitADC.

For large pixels, the circuits and methods described above provide 130dB dynamic range (six and one-half decades) when implemented withtypical values. Although less than for the small pixels, large pixelsare typically operated with a lower gain (10×, or 20 dB, lower), and sothe 130/150 dB combination works in the sense that both pixels willsaturate at approximately the same optical flux levels (light brightnesslevels). In case additional dynamic range is required, the large pixelamplifier 1501 can implement an optional dual gain capability. This iscontrolled by logical signal 1517 from AND gate 1507, also an optionalelement. When both bits b2 and b3 are TRUE, AND gate 1507 output isTRUE, and this is used to enable the optional lower gain range foramplifier 1501.

Output signal chain 1506 receives the large pixel voltage 1510 and allsmall pixel voltages (1511, 1512, and 1513). It contains well-knownfocal plane array elements including switches to select individual pixeloutput voltages and connect them for external output onto connection1535. Signal chain 1506 will also typically contain a buffer amplifier,which ensures that the pixel output voltages do not degrade due toexternal capacitance or resistance in the circuits to which the focalplane array is connected. The signal on connection 1535 is typically ananalog voltage, but it may be a digital value in the case of a digitalfocal plane array. In this case, signal chain 1506 will also containdigitization circuits, such as the well-known sigma-delta circuit.

In alternate embodiments used with digital focal plane arrays, elements1503, 1504, 1505, and 1507 might not be used. In this case, signal chain1506 will contain digitization circuits as described above, which areused in combination with other circuit elements for high dynamic range(HDR) operation (S. Kavusi, K. Ghosh and A. El Gamal, “Architectures forHigh Dynamic Range, High Speed Image Sensor Readout Circuits”,incorporated herein by reference).

Also, in alternate embodiments, some clock signals may be generated onthe ROIC chip, within logic block 1505. For example, φS2 , and φS3 couldbe generated from φS1 by using a high-speed pixel readout clock, acommon element available in all readouts, and on-chip logic.

In the preferred implementation described above, the present inventionuses six clock signals and four logical bits to indicate the gain state,as indicated in FIGS. 12 and 15. In alternate implementations, fewerclock signals and fewer gain state bits may be desirable. Practicalcases of interest include:

-   -   Case 1: Dual fixed clock signals, one fast for large pixels and        one slow for small pixels; and    -   Case 2: Dual clock signals with one small pixel clock variation        so that only φS1, φS2 , and φL1 as shown in FIG. 12 are used.        While not offering the full range of operation of the preferred        implementation, each of these practical implementations still        offers advantages over current focal plane arrays.

For case 1, the dual array can be operated so as to offset theresponsivity range of the large and small pixels, resulting in a largedynamic range through their combined operation. As a specific example,80 micron large pixels with an electronic gain of 20 nV/electron, anintegration time of 10 microseconds, and a full-well capacity of 140million electrons would result in a sensitivity range of approximately2.2 times 10¹³ to 2.4 times 10¹⁷ photons/second for a pixel-fillingoptical flux. 20 micron small pixels with an electronic gain of 320nV/electron, an integration time of 5 milliseconds, and a full-wellcapacity of 8.75 million electrons would result in a sensitivity rangeof approximately 4.4 times 10¹⁰ to 4.8 times 10¹⁴ photons/second forfull pixel optical flux. The degree of offset in responsivity can beadjusted by adjusting the integration times. Times that are more nearlyequal result in less responsivity difference, whereas larger differencesin integration times result in larger responsivity differences. Thissimplified implementation offers the advantage of flexibly extendingdynamic range, but it cannot support two-color radiometry across thefull dynamic range, as the preferred embodiment can.

For case 2, the φS1 and φL1 timing relationships can be set as per theexample given for case 1. Setting φS2 to a shorter integration time,such as 25 microseconds, enables a significantly larger gain range forthe small pixels, from approximately 4.4 times 10¹⁰ to 1.2 times 10¹⁷photons/second for full pixel optical flux. This enables two-colorradiometry for hot events typically associated with weapons firecombined with sensitive detection, fine detail imaging, and fastsensing. Thus, this practical implementation realizes most of thedesired capabilities for military-use and security-use sensor systems.

FIG. 16 depicts the capacitor averaging comparator that is the preferredimplementation for small pixel comparator 1504 (shown in FIG. 15). Forpurposes of discussion, FIG. 16 shows connection to three small pixelsignals: voltages 1511, 1512 and 1513, as depicted in FIG. 15. In actualimplementation, there will be N small pixels connected to the controlcircuit for a single large/small pixel group. Elements 1601, 1602, 1603and 1604 could be implemented as part of signal output chain 1506, orseparately. In operation, switches 1601, 1602 and 1603 operatesequentially. Switch 1601 closes first. Switch 1605 closes immediatelyafterwards and buffer amplifier 1604 drives capacitor 1607 to voltage1511, which is the output of the first small pixel amplifier. Afterthis, switch 1605 opens, then switch 1606 closes. This allows chargetransfer to occur. After this, switch 1606 opens. Then 1602 and 1605close, which charges capacitor 1607 to voltage 1512, the output of thesecond small pixel amplifier and different from voltage 1511. Afterthis, switch 1605 opens, then 1606 closes. This allows a second chargetransfer between the two capacitors. This operation continues until thelast small pixel value (1513) is read. At any time during the sensing ofvoltages 1511, 1512, and 1513 if the voltage on capacitor 1608 risesabove the comparator control voltage, the output of comparator 1609 willlatch to a high value. It will remain high until a small pixel framereset.

In this circuit, the capacitive averaging between 1607 and 1608 operatessimilarly to sub-frame averaging circuits, which are described in U.S.Pat. No. 4,686,373 to Tew et al. and U.S. Pat. No. 5,629,522 to Martinet al., incorporated herein by reference, and in many other documents.However, unlike the previously described uses of the sub-frame averagingcircuit, which averages signals in time, here the circuit is used toaverage multiple pixel values during the same frame time. The relativesize of capacitors 1607 and 1608 and the control voltage 1521 onlatching comparator 1609 determine the operational characteristics ofthis circuit. The ratio of capacitor 1608 to capacitor 1607 will beone-to-one or larger, i.e. two-to-one, three-to-one, etc. Smaller ratiosresult in a faster voltage swing on capacitor 1608 when the voltages forpixels in saturation are sensed. This requires fewer small pixels to bein saturation in order for comparator output 1516 to swing to a high orlogical TRUE value than for larger capacitor ratios. Larger ratiosresult in a slower swing, which will require a greater number of smallpixels to be in saturation in order for output 1516 to swing to a highor TRUE value.

This capacitive averaging comparator circuit works so that if a fewpixels in succession are near saturation, the comparator will latchhigh; however, if the near saturated pixels are not in succession, thenit takes more pixels or a fully saturated value to cause the comparatorto latch high. Thus, it takes a signal in one place (pixels insuccession) to cause a state change, or a brighter signal if the brightarea is diffuse: real, non-clutter signals will tend to be in one place,not scattered.

As described above, preferably the capacitive averaging comparatorcircuit of FIG. 16 provides for weighted averaging based on the ratiosof capacitors 1607 and 1608. By making the second capacitor 1608 larger,a greater number of pixels near saturation are required to result in ahigh value on the second capacitor. Thus, a capacitor ratio can bedesigned based on how many pixels are in a group and the number desiredto be near saturation before the comparator is latched. Typically, itwould be desired that no fewer than 2 or 3 pixels near saturation wouldcause the comparator to be latched: this would be achieved with acapacitive ratio of 1:1 or 1:1.5. However, the capacitive ratio can beincreased to 1:2 or more, if a larger number is desired for designreasons.

FIG. 17 shows one implementation of the large pixel amplifier 1501(shown in FIG. 15) with optional electronic gain control. A CTIA(Capacitive Transimpedance Amplifier) is used. Photocurrent from largepixel 201 is input to op amp 1701. In addition to capacitor 1702, thestandard integration capacitor for the CTIA, a second and largercapacitor, 1703, is also included in the circuit. Switches 1704, 1705,and 1706 control this second capacitor based on the optional gain signal1517. If 1517 is high (logical TRUE), switches 1704 and 1705 are onwhile 1706 is off due to inverter 1707. This adds capacitor 1703 intothe circuit, adding its capacitance to that of 1702, and reducing theamplifier's electronic gain. If the gain signal 1517 is low (logicalFALSE), switches 1704 and 1705 are off, and switch 1706 is on, keepingcapacitor 1703 at a near zero voltage. Large pixel clock φL controls thereset switch 1708. It turns on for reset ensuring that voltage on1703—and 1704 if used—will be driven to near zero.

In any preferred implementation, the small pixel amplifier 1502 (shownin FIG. 15) is a simple amplifier with no elements to enable additionalgain control. The small pixel amplifier can take multiple forms such asbuffered direct input, CTIA, and so forth.

The presently preferred embodiment of the invention utilizes specialmethods to perform two-color radiometry, which enable higher accuracyresults. In conventional two-color cameras, detection is enhanced bydetermining simple pixel brightness relationships (R. Waterman, “TwoColor IRFPAs for Navy Missile Warning”, Night Operations Symposium, 13Mar. 2002, NDIA, Arlington, Va. Accession number ADM002270, incorporatedherein by reference). For example, color-one is brighter than color-two,or they are approximately equal. In the present invention, new methodsenhance accuracy and account for differences between the large and smallpixels.

In a typical array, the first video processing step is non-uniformitycorrection (NUC). This well-known technique normalizes data to accountfor the inherent differences in response between pixels to the sameoptical input; some pixels exhibit higher gain, some lower, and thereare difference in offset, the signal present with no illumination. Afterthis, detection methods utilize temporal and spatial methods thatidentify likely events. These detection methods require that the systemcontinuously determine statistical characteristics of the typical scene,for example, the average brightness and variation in brightness. Thisinformation is used to determine when something unusual has occurred,such as a bright flash of light that exceeds the average by astatistically significant amount. Then, the system uses two-colormethods to determine if the unusual event is likely to be a threat. Forexample, a flash of light observed from an airborne platform, which isbrighter than the surrounding background by a statistically significantamount, could be a missile launch. If the two-color test indicates atemperature near 5800 degrees Kelvin, the bright flash is a solar glintand can be dismissed, whereas if the two-color test is consistent with atemperature in the 1000 to 2000 degree Kelvin range, it may be a missilelaunch.

These steps can be summed up as follows:

Non-Uniformity Correction (NUC). This corrects for variation in pixelresponse so that when the system views an optically flat background(same color, same brightness), it produces an optically flat image.

Determine Background. Average background brightness and variation isdetermined in both colors using well-known spatial-temporal techniques,which include pixel averages and variation over time, and determiningaverages and variations across small areas (groups of pixels).

Detect Event. Events are detected using well-known statistical methods,for example, whenever a short-term average exceeds a long-term averageby a pre-designated number of standard deviations. Pixels are typicallyprocessed in small contiguous groups, such as 3×3 pixel groups.

Two-Color Tests. These determine whether events detected in the DetectEvent step are real threats based on an approximate temperaturedetermination.

For the dual array, the first two steps—NUC and Determine Background—areperformed as for standard arrays. After these first two steps, theprocess differs. First, two types of spatial-temporal detectionprocessing is performed:

Detect Fast Events. Large pixel data is processed with algorithms thatdetect events that occur within a specified time range, such as a totalevent time of 2 milliseconds. Typically, individual pixel data is usedfor the fast event processing rather than pixel groups.

Detect Standard Events. Small pixel data is processed using well-knownmethods typically used in standard arrays. Typically, small groups ofpixels are used. For the dual array, these groups are of a differentsize than the small pixel sub-array associated with a large pixel.

Once either a fast or standard event is detected, two-color methods areused to determine if it is a real threat. However, the pixel data mustbe normalization for proper comparison due to the different electronicgains, integration times, and spatial IFOV's (the pixel InstantaneousField of View). Explicit adjustments are made due to the presence ofbackground light, i.e. that part of the scene excluding the event ofinterest. The normalization processing consists of corrections forspatial and temporal factors, then adjusting for pixel responsivitydifferences.

Correct Large Pixel Response for Spatial Extent. FIG. 18 shows theexample of event 1801 on large pixel 1806. The spatial extent of event1801 is much smaller than pixel 1806, so the total response from 1806will consist of event 1801 and background light. Therefore, thebackground average value, weighted to account for the spatial extent ofthe event, is subtracted from the overall large pixel response todetermine the response to event 1801 alone. The small pixel response isused to determine the spatial extent of 1801 and what the weightingvalue should be. In this case, event 1801 covers about 8% of the largepixel, so a weighting factor of 0.92 would be used. In this example,event 1801 is very small, and this indicates that the spatial extent ofevent 1801 is defined by the optical diffraction limit of the system.Therefore, the weighting factor would be adjusted in consideration ofthe wavelength of the two color bands. For example, if the small pixelsrespond with a center wavelength of 8 microns, and the large pixelsresponse with a center wavelength of 4 microns, a weighting factor of0.96 would be used. For events with larger spatial extents or forclosely spaced spectral bands, this adjustment to the weighting factorfor wavelength difference is not necessary. For events that fill theentire large pixel, no correction for spatial extent is necessary.

Correct Small Pixel Data for Temporal Extent. FIG. 19 shows the exampleof an event with duration 1901. Also shown are the small pixelintegration time 1202 (see FIG. 12) and the large pixel integration time1212 (see FIG. 12). This event has a duration less than time 1202, andso the total small pixel response will consist of background, which isintegrated during the time periods indicated by 1902 and 1903, as wellas the signal of interest, which is integrated during the time periodindicated by 1904. Based on these time durations, the small pixelbackground average, weighted based on time periods 1904 and 1902 plus1903, is subtracted from the total pixel response to determine theresponse due to the event of interest. In this example, the eventduration is 10 time periods 1212, and the background illumination periodis 20 time periods 1212. Therefore, the weighting factor would be 0.66,and this adjusted background would be subtracted from the overall smallpixel response.

Correct for Pixel Gain. The above two steps adjust for two out of thethree necessary factors: the different integration times used by largeand small pixels and their different spatial extent, which is equivalentto adjusting for their different sizes. The last factor is electronicgain, which is a fixed factor set by design unless auxiliary gain isused in the large pixel. The electronic gain is the transimpedance gainof the pixel amplifier, and the correction factor is simply the ratiobetween large pixel amplifiers and small pixel amplifiers.

Once the two-color data has been normalized with these three steps,color ratios can be determined. In FIG. 18, pixels 1802, 1803, 1804, and1805 would be added together, and this total response would be comparedto large pixel 1806. Alternatively, each small pixel could be comparedindividually; this would require a secondary normalization to accountfor pixel size. In FIG. 19, time periods 1212 within event duration 1901would be added together and compared to the adjusted small pixelresponse. Alternatively, large pixel data from time periods 1212 withintime period 1905, which might represent peak intensity of the signal ofinterest, could be compared with the adjusted small pixel response. Thiswould require a secondary normalization to account for the shorter timeperiod 1095. Any of these two-color ratios could be used in existing ornew algorithms for threat determination.

Because the dual array two-color processing explicitly accounts forspatial and temporal extent of the signal of interest and removesinvalid background light, it provides more accurate radiometric measuresthan standard arrays. It also provides enhanced flexibility by enablinga variety of peak signal or average signal measurements.

While the present invention is being disclosed in connection with thepresently preferred embodiment described herein, it should be understoodthat there might be other embodiments that fall within this spirit andscope of the invention as defined by the claims. Accordingly, nolimitations are to be implied or inferred in this invention except asspecifically and explicitly set forth in the claims.

INDUSTRIAL APPLICABILITY

This invention can be used any time it is necessary or desirable toachieve a sensor with the ability to sense very high-speed events,produce a high-resolution image, support a high (large) dynamic range,and avoid false alarms from manmade signals or clutter.

What is claimed is:
 1. A capacitor averaging comparator for comparingelectrical signals from a plurality of pixels, each electrical signalhaving a signal voltage comprising: a plurality of signal switches, eachsignal switch corresponding to one of said electrical signals; a bufferamplifier connected to said signal switches; a first capacitor; a firstcapacitor switch connected between said buffer amplifier and said firstcapacitor; a second capacitor; a second capacitor switch connectedbetween said first capacitor and said second capacitor; and a latchingcomparator connected to said second capacitor controlled by a comparatorcontrol voltage; a clock operably connected to said pixels to read saidelectrical signals at a clock rate having an integration time and areset time; wherein when said signal switches are sequentially closed,each of said electrical signals is sequentially sent to said bufferamplifier, said first capacitor switch is closed, said buffer amplifierdrives said first capacitor to store a first stored voltage that isproportional to said signal voltage, said first capacitor switch opens,said second capacitor switch closes and charge is transferred betweensaid first and second capacitors, causing any prior stored voltage onsaid second capacitor and said first stored voltage on said firstcapacitor to form a weighted average voltage, and said comparatorcompares said weighted average voltage to said comparator controlvoltage; wherein said weighted average voltage is based on the ratio ofcapacitance between said first capacitor and said second capacitor, andwherein if any of said weighted average voltage for a preset number ofpixels exceeds said comparator control voltage, then said latchingcomparator latches to a high value indicating said preset number ofpixels are saturated; wherein if said weighted average voltage indicatesthat said preset number of pixels are saturated, said clock changes saidclock rate to have a changed integration time and a changed reset time,wherein said changed integration time is less than said integration timeand said changed reset time is greater than said reset time, and whereinif said weighted average voltage indicates that less than said presetnumber of pixels are unsaturated, said clock maintains said integrationtime and said reset time; wherein if said weighted average voltageindicates that a preset number of pixels are saturated while using saidchanged integration time and said changed reset time, said clock changessaid clock rate to have a re-changed integration time and a re-changedreset time, wherein said re-changed integration time is shorter thansaid changed integration time and said re-changed reset time is longerthan said changed reset time, and wherein if said weighted averagevoltage indicates that less than said preset number of pixels areunsaturated while using said changed integration time and said changedreset time, said clock rate will remain at said changed integration timeand said changed reset time; whereby the reading of said electricalsignals at said changed integration time and said re-changed integrationtime, extends dynamic range of said plurality of pixels.